1. Field of the Invention
This invention relates generally to methods of distributing clock signals in computer systems. In particular, the present invention provides a method and apparatus for distributing an optical clock signal in computer systems.
2. Background Information
In computer systems, a system clock signal is distributed throughout as a reference signal to control the timing of events. The use of a reference clock simplifies the design of data transfer structures within the computer since they can be designed to operate synchronous to the system clock. Synchronous designs are easier to conceptualize and to partition.
System clock signals are typically distributed from a single source point to various destination points within the computer system, which may be located some distance apart. In complex, high performance data processing systems such as those manufactured by Cray Research, Inc., the Assignee of the present invention, the clock frequency is high and the number of points to which the system clock is provided may be large. In prior art systems, high frequency clock signals were typically distributed as electrical signals sent point-to-point from the source to the destination. Point-to-point connections made in this manner act as individual antennae radiating radio frequency (RF) energy that adds to system noise and can significantly affect reliability.
Also, for reasons which will be discussed later, signals distributed by this means do not arrive at all destination points at exactly the same time. The difference in time between these arrivals is called skew.
Within a computer system, data is passed from register to register, with varying amounts of processing performed between registers. Registers store data present at their inputs either at a system clock transition or during a phase of the system clock. Skew in the system clock signal impacts register-to-register transfers; it may cause a register to store data either before it has become valid or after it is no longer valid.
On slower computer systems, skew is usually a small portion of the clock period. Its impact can be reduced by adding delay to the data path. However, on high performance data processing systems such as those manufactured by Cray Research, Inc., the Assignee of the present invention, the same amount of skew may be a substantial portion of the clock period and may actually limit the speed at which the computer system can operate. The amount of delay that can be added to a data path is limited. Therefore skew becomes a major design factor in the transfer of data.
Clock skew is caused by a number of factors. A typical path for a clock signal will include interconnections between circuit boards, fanout gates, circuit board foil paths, and integrated circuit (IC) interconnect metal. Each of these provides an opportunity for introducing undesired clock skew. The amount of time it takes a signal to travel along a wire, foil path, or interconnect metal is called its electrical path length and is dependent upon physical length and capacitance. All else being equal, a signal will take longer to travel a long path than a short one. If the electrical path lengths of all the clock signal paths are not equal, skew is introduced.
A second major source of skew is from the integrated circuits that drive the clock signals. In a typical system the clock signal is distributed to a limited number of clock fanout devices on a circuit board. These clock fanout devices in turn distribute the clock signal to the remaining integrated circuits on the board. There will always be differences in propagation delay between fanout devices. One reason is variation in the semiconductor manufacturing process; another is temperature and voltage differences. Propagation delay differences between fanout devices further increase clock skew. In addition, in systems where there are a large number of integrated circuits, two or more levels of fanout devices may be needed to provide a clock signal to all the registers in the system. Each level of fanout has the potential of adding additional skew to the system clock signal.
Crosstalk from adjacent signals, coupled RF interference and power and ground noise also act to increase clock skew. For example, if a signal's voltage level is altered by crosstalk, then the point in time when the signal is determined to have switched will be altered, thus introducing skew.
As system clock periods shrink there is increasing pressure on the computer architect to reduce in determinism in the system design. Clock skew, like setup and hold time and propagation delay, increase the amount of time that data is in an indeterminable state. System designers must be careful that this indeterminable state does not fall within the sampling window of a register if they are going to preserve data integrity. In prior art systems, data path delay was often used to move the location of the indeterminable state outside the sampling window. However, as clock frequencies approach 500 MHz, the ability to position the indeterminable state becomes more difficult. As a result clock skew reduces the ability to increase the system clock frequency.
Several techniques have been used to reduce clock skew. System designers attempt to equalize the wire, foil path, and interconnect metal lengths between the clock source and all destinations. This helps to reduce variation in the electrical path length due to physical length. However, since each path may have a different impedance, it is difficult to truly match electrical path length.
Methods are well known in the art for minimizing crosstalk, shielding signals and providing more stable temperature and voltage references. Likewise, variation in propagation delay between fanout devices can be controlled to a degree. This can be done by mandating stringent screening requirements or by matching components during assembly. However the former drives up the price of the devices while the latter drives up manufacturing costs. Truly matched performance is difficult to achieve because of the different loads and impedances faced by each device.
Clock distribution network tuning is a different approach at reducing the effects of clock skew. Delay is added selectively to clock paths in an attempt to equalize the delay through each of the paths. A representative tuning strategy is disclosed in the co-pending and commonly assigned patent application Ser. No. 07/465,947 filed Jan. 16, 1990 by Stephen E. Nelson et al. entitled "CLOCK DISTRIBUTION SYSTEM AND METHOD", which application is incorporated herein by reference. That application discloses the use of selectable delay paths to tune clock signal paths to compensate for clock skew.
In addition, optical clock distribution networks have been proposed as a viable alternative to electrical clock distribution. Optical fiber as a transmission medium provides numerous advantages. An optical clock distribution method provides a noise-free signal transmission environment, is resistant to electromagnetic interference, and supports high transmission rates. However, in systems proposed to date, high frequency clocks with minimal skew have been difficult to achieve.
The effectiveness and practicality of these methods varies. It is clear that there has existed a long and unfilled need in the prior art for a clock distribution method and apparatus capable of reducing clock skew. The present invention solves these and other shortcomings of the techniques known in the prior art.